Papers with token classification
VILA: Improving Structured Content Extraction from Scientific PDFs Using Visual Layout Groups (2022.tacl-1)
Copied to clipboard
| Challenge: | Recent work has improved extraction accuracy by incorporating elementary layout information, for example, each token’s 2D position on the page, into language model pretraining. |
| Approach: | They propose a method that explicitly models VIsual LAyout (VILA) groups, that is, text lines or text blocks, to further improve extraction accuracy. |
| Outcome: | The proposed methods show that inserting special tokens denoting layout group boundaries can lead to a 1.9% Macro F1 improvement in token classification. |
Active Learning Design Choices for NER with Transformers (2024.lrec-main)
Copied to clipboard
Robert Vacareanu, Enrique Noriega-Atala, Gus Hahn-Powell, Marco A. Valenzuela-Escarcega, Mihai Surdeanu
| Challenge: | In the field of natural language processing, active learning is a technique that is used to decide which examples are worth annotating . a number of studies have focused on sequence classification, text classification, question answering, and question answering. |
| Approach: | They propose two different approaches to deal with partially-annotated sentences . they propose an annotation scheme that can be used to train with tokens . |
| Outcome: | The proposed approaches achieve comparable or better performance than sentence-level annotations with a smaller number of annotated tokens. |
AutoTrain: No-code training for state-of-the-art models (2024.emnlp-demo)
Copied to clipboard
| Challenge: | AutoTrain is an open-source, no code tool/library which can be used to train models on custom datasets. |
| Approach: | They propose an open-source, no-code tool/library to train models on custom datasets. |
| Outcome: | The open-source, no-code tool/library can be used to train models on custom datasets. |
Knowledge Augmentation Enhances Token Classification for Recipe Understanding (2026.eacl-long)
Copied to clipboard
| Challenge: | Using entity type-specific and knowledge-augmented token classification, we achieve state-of-the-art (SOTA) results on 5 out of 7 benchmark recipe datasets, significantly outperforming traditional token classification methods. |
| Approach: | They propose an entity type-specific and knowledge-augmented token classification framework to improve encoder models’ performance on recipe texts. |
| Outcome: | The proposed model outperforms traditional token classification methods on 5 out of 7 recipe datasets and is the largest annotated food-related dataset to date. |
Automatic Punctuation Model for Spanish Live Transcriptions (2024.lrec-main)
Copied to clipboard
Mario Perez-Enriquez, Jose Manuel Masiello-Ruiz, Jose Luis Lopez-Cuadrado, Israel Gonzalez-Carrasco, Paloma Martinez-Fernandez, Belen Ruiz-Mezcua
| Challenge: | Punctuation marks, including periods, commas, question marks, and exclamation points, are crucial cues for sentence boundaries, intonation, and context, significantly aiding in the comprehension of spoken Spanish. |
| Approach: | They propose to integrate punctuation into automatic transcription tools by using a model pre-trained with data from the Spanish National Library. |
| Outcome: | The proposed model improves readability and accuracy of Spanish transcriptions by incorporating punctuation into the outputs generated by automatic transcription tools. |
On Long-Tailed Phenomena in Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | State-of-the-art Neural Machine Translation models struggle with generating low-frequency tokens, tackling which remains a major challenge. |
| Approach: | They propose a loss function to better adapt model training to structural dependencies of conditional text generation by incorporating inductive biases of beam search into the training process. |
| Outcome: | The proposed method leads to significant gains over cross-entropy across different language pairs, especially on the generation of low-frequency words. |
Transforming Term Extraction: Transformer-Based Approaches to Multilingual Term Extraction Across Domains (2021.findings-acl)
Copied to clipboard
| Challenge: | Automated Term Extraction (ATE) is a challenging task, with few exceptions. |
| Approach: | They propose to use a transformer-based term extraction model to extract terms from sentences . they also propose to employ a language model for token classification and a sequence model to reduce sentences to terms . |
| Outcome: | The proposed models outperform baselines on the ATE challenge TermEval 2020 dataset in English, French, and Dutch. |
Transfer-Free Data-Efficient Multilingual Slot Labeling (2023.emnlp-main)
Copied to clipboard
| Challenge: | Slot labeling (SL) is a key component of task-oriented dialogue systems . extending the system to any new language-domain-task configuration requires expensive data annotation . |
| Approach: | They propose a two-stage slot labeling approach which transforms sentence encoders into effective slot labels. |
| Outcome: | The proposed approach is especially effective for the most challenging transfer-free few-shot setups. |
Lazy-k Decoding: Constrained Decoding for Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Specifically, we combine probabilistic models with constrained decoding approaches in structured prediction tasks. |
| Approach: | They propose a constrained decoding method called Lazy-k to combine probabilistic models with constrained methods in structured prediction. |
| Outcome: | The proposed method allows for more flexibility between decoding time and accuracy. |
Just Pass Twice: Efficient Token Classification with LLMs for Zero-Shot NER (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to name entity recognition use autoregressive decoding, hallucinated entities, and formatting errors. |
| Approach: | They propose a method that allows causal LLMs to perform discriminative token classification with full bidirectional context. |
| Outcome: | The proposed method surpasses the previous best method on zero-shot NER benchmarks by +7.9 F1 on average across CrossNER and MIT benchmarks. |
Nested Named Entity Recognition as Single-Pass Sequence Labeling (2025.findings-emnlp)
Copied to clipboard
| Challenge: | nested named entity recognition is a sequence labeling task that can be trained using any off-the-shelf sequence labelling library. |
| Approach: | They use prior work that linearizes constituency structures to create a nested named entity recognition task. |
| Outcome: | The proposed method reduces the complexity of the predicted nested entity recognition problem to a simple token classification task. |
Event Extraction in Basque: Typologically Motivated Cross-Lingual Transfer-Learning Analysis (2024.lrec-main)
Copied to clipboard
| Challenge: | Using a multilingual language model, Event Extraction tasks require humans to follow complicated guidelines and follow complicated rules. |
| Approach: | They propose a multilingual multilingual language model that is trained in a source language and applied to a target language. |
| Outcome: | The proposed model is based on a multilingual event extraction dataset for Basque . it shows that the shared linguistic characteristic between source and target languages does have an impact on transfer quality. |
Improving Text Readability through Segmentation into Rheses (2024.lrec-main)
Copied to clipboard
| Challenge: | a new study examines the segmentation of sentences into rheses to improve readability for dyslexics . short lines of text can be beneficial for dyslexia sufferers as it limits attention span . however, random line splits can be confusing than helpful . |
| Approach: | They propose to segment sentences into rhythmic and semantic units to improve comprehension . they also use a bilingual dataset to evaluate the efficiency of their approach . |
| Outcome: | The proposed approach achieves an F1 score of 90.0% in English and 91.3% in French . the proposed approach also demonstrates the potential of leveraging prosodic elements . |
Token-Level Metrics for Detecting Incorrect Gold Annotations in Named Entity Recognition (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Annotated datasets for supervised learning often contain incorrect labels, i.e. label noise. |
| Approach: | They compare popular sample metrics for detecting incorrect annotations in named entity recognition (NER) they find that training dynamics metrics work the best overall, and they detect errors that the model has not yet memorized . |
| Outcome: | The proposed measures reduce label noise across noise types by detecting errors in trained models. |
JL-Hate: An Annotated Dataset for Joint Learning of Hate Speech and Target Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing data resources for the detection of hate speech focus on text sequence classification, but the target of hateful content is lacking. |
| Approach: | They propose a tweet dataset for the task of joint learning of hate speech detection and target detection called JL-Hate. |
| Outcome: | The proposed dataset performs similar tasks to the existing datasets in sequence and token classification tasks. |
Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Prediction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in multimodal pre-trained models have significantly improved information extraction from visually-rich documents (VrDs). |
| Approach: | They propose a method to predict token sequences within visually-rich documents by a simple prediction head. |
| Outcome: | The proposed method can be used to predict token mentions as token sequences within documents. |
FLEXITOKENS: Flexible Tokenization for Evolving Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Widely used subword tokenizers overfragment sequences in unseen domains, languages, and scripts . inefficient tokenizer models can cause overfragments in out-of-distribution domains if not trained properly . |
| Approach: | They propose a byte-level LM with learnable tokenizers to make tokenization adaptive . they propose 'flexitoken' which enables significantly greater flexibility during adaptation . |
| Outcome: | The proposed method significantly reduces token overfragmentation and improves on multilingual benchmarks and domains. |
Nested Noun Phrase Identification Using BERT (2024.lrec-main)
Copied to clipboard
| Challenge: | a number of methods for identifying noun phrases have been developed . chunking-like methods do not represent the fact that noun phrase can be nested. |
| Approach: | They propose a method of finding all noun phrases in a sentence nested to an arbitrary depth using the BERT model for token classification. |
| Outcome: | The proposed method achieves very good results for both Swedish and English . it is based on the BERT model for token classification . |
TransAlign: Machine Translation Encoders are Strong Word Aligners, Too (2025.findings-emnlp)
Copied to clipboard
| Challenge: | translation-based approaches to cross-lingual transfer (XLT) are limited. |
| Approach: | They propose a word aligner that utilizes the encoder of a massively multilingual MT model. |
| Outcome: | The proposed word aligner outperforms existing WA and state-of-the-art non-WA-based methods in token classification tasks. |